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ABSTRACT 

This study presents a conceptual framework for supporting mobile peer assessment by incorporating augmented reality 
technology to eliminate limitation of reviewing and assessing. According to the characteristics of mobile technology and 
augmented reality, students’ work can be shown in various ways by considering the locations and situations. This study 
proposes a novel mobile peer-assessment system which incorporates augmented reality into the reviewing and assessing 
processes. The mechanism enables students to enhance work interpretation, frequently interact with peers, represent their 
thinking and reflect upon their own works. Moreover, the mobile AR technique provides personalized and location-based 
adaptive contents that enable individual students to interact with the mixed reality environment and to observe how works 
are possibly applied to the real world in the future. The overall process facilitates students in reviewing works based on 
various dimensions, acquiring important knowledge, fostering critical thinking skills and reflection as well as promoting 
meaningful learning. 
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1. INTRODUCTION 

Peer assessment has become increasingly popular in education due to the support of group learning and the 
enhancement of learning effectiveness. In peer assessment process, students participate cognitive activities 
including doing assignments, devising assessment criteria, reviewing, summarizing, clarifying, providing 
feedback, diagnosing errors, identifying missing knowledge or deviations and evaluating the quality of peers’ 
work (Van Lehn, et al., 1995; Liu, et al., 1999; Sitthiworachart & Joy, 2003). The majority of previous studies 
emphasize conditions, methods and outcomes of peer assessment and focus on the quality of students’ work, 
domain-specific skill and peer assessment skill for outcomes (Van Zundert, et al., 2010). In recent years, 
mobile technology provides the potential of creating innovation learning experiences. Students can acquire 
learning materials, share ideas, and construct knowledge anytime anywhere by using their own handheld 
devices. In order to eliminate the limitation of space and time, mobile peer assessment positively influences the 
assessment methods and outcomes and enables students to submit their own work, review peers’ work, mark 
and provide feedback conveniently. 

However, during the peer assessment process, providing students with sufficient information to review 
peers’ work is preferable. Augmented reality (AR) is the technology that provides the right contents at the right 
place at the right time. The mobile AR technique is able to overlay virtual objects on the real work to present 
rich information to students and construct meaningful presentation by combining location-awareness and 
contextual learning. There is a positive relation between providing students with the opportunity to review 
peers’ work based on the mobile AR technique and reflecting upon their own work. 

In order to provide full insight into effective peer assessment processes, issues regarding content 
presentation as well as assessment methods require more attention. 
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Therefore, this study presents a conceptual framework for providing intelligent and mobile supports 
through incorporating the AR technique to enhance work presentation and the effectiveness of peer assessment. 
In this framework, students are able to review peers' work by using various dimensions and receive assessment 
results immediately. The difficulty of reviewing peers' work and understanding peers' thinking can be resolved 
and sufficient information representation enables accurate assessment. Most importantly, appropriate 
assessment criteria and rich feedback facilitate students to reflect upon their own work and improve the quality 
of their work. 


2. PEER ASSESSMENT IN MOBILE LEARNING 

Peer assessment has been widely recognized as an educational arrangement in which students assess peers’ 
work and provide feedback (Van den Berg, et ak, 2006), as well as a learning tool for improving student’s 
performance in collaborative learning environment (Topping, et ak, 2000). Various studies related to 
education, business, health and science on self and peer assessment in higher education have been proposed 
(Searby Ewers, 1997; Ballantyne, et ak, 2002; Prins, et ak, 2005; Price & O’Donovan, 2006). These studies 
reveal that students who involve in the interactive assessment process can enhance their interpretation and 
reflection. Regarding how to effectively involve students in peer assessment, these processes including 
exploration of assessment criteria, presentation of works, assessment methods, coordination of assessment and 
feedback are very critical (Chen, 2010 ; Lan, et ak, 2012). Most studies focus on the conditions, methods and 
outcomes (Van Zundert, et ak, 2010) and have proposed computerized-based peer assessment systems to 
support the assessment process (Davies, 2000; Lin, et ak, 2001). Appropriate technology applied in peer 
assessment can assist the reviewing and assessing activities. Computer networks facilitate students to 
participate in assessment activities anytime anywhere and enable teachers to review assessment progress. 
On-line peer assessment systems that can do away with conditions restricting various assessment activities in 
classrooms can eliminate the time and the cost in communicating with each other and printing out student work 
or assessment forms. 

In recent years, students attempt to learn in various locations, and therefore mobile learning is becoming 
widespread. Mobile technology provides the potential of creating innovative learning experiences that can take 
place anytime and anywhere (Shih, 2010). Because of the characteristics of mobile technology such as 
ubiquity, smaller size, comparative affordability, and the prevalence of wireless networks, more and more 
researchers have developed application on handheld devices such as mobile phones, tablet computers and 
PDAs to support learning activities. Some studies have proposed the critical issue of how to use handheld 
devices to enhance assessment (Penuel, et ak, 2007; Shin, et ak, 2007). Students can use handheld devices to 
flexibly conduct project-based learning and self-assessment inside and outside classrooms. A few researchers 
have reported the findings about how to use mobile technology for self- and peer-assessment (Chen, 2010). 
Chen indicated that combining mobile technology with the concept of round-table presentations, the mobile 
self- and peer-assessment system can assist teachers to arrange assessment activities more flexibly and make 
students more attentive to presentation, interaction and feedback in the assessment process. However, most of 
these studies emphasize the exploration of assessment criteria, marking process and the promotion of feedback 
to enhance the effectiveness and reflection of self- and peer assessment. Actually, it is a very critical issue that 
students’ work can be presented in detail during the assessing process. Through reviewing peers’ work, 
students can understand how to mark and reflect upon their own work. 

According to the characteristics of mobile technology, students’ work can be shown in various ways by 
considering the locations and situations; moreover students can communicate with peers as well as observe 
peers’ work anytime anywhere. This study proposes a novel mobile peer-assessment system which 
incorporates augmented reality into the reviewing and assessing process. The mechanism enables students to 
enhance work interpretation, frequently interact with peers, represent their thinking and reflect upon their 
work. Through the reviewing and interactive process, assessment accuracy and quality can be improved. The 
overall process facilitates students in fostering critical thinking skills and reflection as well as promoting 
meaningful learning. 
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3. PEER ASSESSMENT WITH MOBILE AUGMENTED REALITY 

Augmented reality (AR) is the technology that shows the right contents for the right device to the proper person 
at the right place and at the right time (Chang & Tan, 2010; Chan, et ah, 2010). It can overlay virtual objects on 
the real world to fulfill the feeling of immersion and therefore supplements user’s everyday life with 
information, images, sounds, and other sensory information from their device. Shortly to say, through putting a 
virtual layer of information over the real world, AR pretends that virtual objects are real and presented at the 
right place. The widely accepted definition of AR is “Augmented Reality allows the user to see the real world, 
with virtual objects superimposed upon or composited with the real world. AR supplements reality, rather than 
completely replacing it.”(Azuma, 1997; Azuma, et ah, 2001). AR is thought to present certain advantages over 
more traditional ways of accessing information (Anastassova, 2007). 

Alongside mobility, development of positioning technologies make it possible to keep track of students 
and provide them with tailored learning contents based on their real-time locations. Furthermore, 
location-based e-learning provides a personalized learning experience and helps in keeping the students 
engaged in the learning activities and enhancing their effectiveness (Chen, et ah, 2007). Previous studies have 
indicated that the combination of location-awareness and a contextual learning approach can enable students to 
better construct meaningful contextualization of concepts (Michie, 1998; Patten, et ah, 2006). For the purpose 
of locating virtual information at the right place in real word, tags or markers are necessary for recognition. AR 
recognizes the tag and gets its position as the position of the corresponded virtual information. There are two 
types of tags: one is so called “AR ToolKit marker”. It’s a monochrome graph surrounded by a square frame. 
The other one is full-on image recognition. The square frame of “AR ToolKit marker” transforms to a 
parallelogram when it be projected on the screen. By reversing this procedure of mapping a parallelogram to a 
square, the position and direction of the square frame in the real world can be detected, and then the virtual 
object information can be overlaid on the screen of the handheld device. However, real objects can be observed 
in various dimensions, but it is more difficult to recognize real objects than “AR ToolKit marker”. Currently, it 
is still not quite at the stage of full-on image recognition, but other researchers are working on it. Because real 
object recognition does not need extra tags, it would become the most popular approach in the near future. 

Few years ago, if someone wanted to show virtual information on real objects, he needed to wear some 
machines on his neck. These machines include one camera to capture images of the real word, a wireless 
network transporter to send images to computer systems and get feedback information that merges virtual 
objects with real-world images seamlessly, and a projector to show virtual information on real objects. 
However, it is not convenient at all. In recent years, significant advancements related to wireless and mobile 
technologies make handheld devices which combine several utilities to be the most convenient platform for the 
AR technique. The camera on the handheld device can capture images of real world, a compass can detect the 
direction of user’s face, the GPS receiver can locate the position of users, and the monitor can show the images 
of the outcome that the mobile AR technique create, which can be text, table, image, video and their 
combination. Even more, extra components such as buttons or tables can be included for interaction. The 
mobile AR technique provides pliable mobility and a location independent service without constraining the 
individual to a specific area. According to the NMC Horizon Report 2012 K-12 Edition, AR supports visual 
and highly interactive forms of learning in education. Students can use it to construct new understanding based 
on interactions with virtual objects that bring underlying data to life as it responds to user input (NMC Horizon 
Report, 2012). Numerous researches has proposed that the AR technique can help students to learn in serious 
games, language learning, e-books, storytelling, driving guidance, and so on (Azuma, 1997; Van, et ah, 2010; 
Chen (& Tsai, 2010). By this way, AR holds the possibility to revolutionize the way in which information is 
demonstrated to people and has great potential for on-demand, context-aware, and collaborative training 
(Hollerer, 2001). Moreover, the mobile AR technique provides personalized and location-based adaptive 
contents for individual students to interact with the mobile viewing environment, and see how works are 
applied to the real environment in the future at the current place. 

According to the above mentioned, the mobile AR technique can obviously support students to review 
peers’ work during the peer assessment process. Eormerly, in a design course, students only review the work 
based on assessment criteria such as originality, produce skill, colour scheme and so on but cannot view the 
usability of the work in the future in this environment where the assessor located. The most important 
functionality of a location-based mobile AR technique is to provide the proper contents according to students’ 
current location. 
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The relevant applied contents in students’ vicinity will be presented by the mobile AR technique 
automatically while students walk in the area. For example, how a painting can be hanged on the wall or 
become a fresco or how a handiwork will be if it is rebuild to a sculpture putting in this environment. "The 
incorporation of various rich sensors into new phones such as GPS location, wireless sensitivity, compass 
direction, accelerometer movement as well as sound and image recognition is enabling new ways in which we 
are able to interact with the world around us." (Nokia Research Center, 2009). The mobile AR technique can 
fuse digital media with the physical world to create the proper conditions for locative, contextual and 
situation-based demo scenarios. In this study, during the peer assessment process, assessors not only assess the 
works presented in front of them but also view the future application of target works. Therefore, assessors can 
judge the design skill of designers as well as the usability of the work in the future. 


4. SYSTEM REALIZATION AND ILLUSTRATIVE EXAMPLE 


4.1 System Architecture 

Augmented reality is defined as a real-world environment whose elements are built upon computer -generated 
sensory input such as sound, video, graphics or GPS data. In the educational field, there are many situations 
cannot be experienced in the classroom. Augmented reality is the latest technology that can accommodate or 
modify their learning experience to their specific needs. So what AR allows us to do is to see virtual objects in 
a real world environment with the aid of camera and some display devices (monitor or head mounted display). 
Subsequently, the procedure of peer assessment enriched by the mobile AR technique is described. This 
procedure shows how AR can enhance the effectiveness of reviewing and assessing during peer assessment. 
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Figure 1 . The Architecture of the Mobile AR Technique 
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The architecture of the mobile AR technique includes three parts such as hardware repository, persistent 
storage, and works demonstration as shown in Figure 1 . In hardware repository, 2d/3d unique tags which are 
able to recognize and present virtual objects in right locations are necessary and mobile displays are included. 
Students’ works and expositions of design about these works are categorized into persistent storage. These 
materials provide extra information of virtual objects. In the part of work demonstration, the result that overlay 
virtual object images on real object images can be presented on handheld devices such as mobile phones or 
tablets. Through incorporating the mobile AR technique into peer assessment. Figure 2 shows the framework 
of the mobile-AR peer-assessment system (MARFAS). 



Figure 2. The Framework Of The Mobile Augmented Reality Peer Assessment System 

There are three databases in the cloud, including the student profiles, the AR and virtual object database and 
the assessment database in Figure 2. At the beginning of the assessment activities, students must login to 
authenticate their identities. All data related to students have been built in the user database. Subsequently, the 
target work shows up in front of assessors, the system goes on getting all information including assessors’ 
location, the direction that they face and the situation such as indoor or outdoor. These local data is collected by 
handheld devices and sent to the system. During the peer assessment process, all procedures are parted into 
three modules including the authentication module, the context aware module, and the AR interactive module. 
The authentication module enables right people to get right information to assess right works. The context 
aware module enables assessors to use right device to receive right context for assessment, and the AR 
interactive module enables assessors to review peers’ works conveniently and intuitively such that the 
assessment can be more diversified and every assessee learns more from other works. In the context aware 
module, the system judges these local data and then choices a proper context for the assessor from the virtual 
object database. All data are ready for the AR technique to overlay on the real world image, and thus assessors 
mark these works more conveniently and accurately. 
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4.2 Walk-through Illustrative Example 

Peer assessment was facilitated by working in small groups of three to four students. These students were better 
able to compare feedback from different peers to determine its relevance (Van Zundert, et ak, 2010). Therefore, 
in this study, 50 undergraduates major in visual communication design enroll a fundamental design course and 
are arranged in small groups. The teacher assigns a painting work and then students can receive the notification 
on their own mobile phone or tablet. Students have three weeks to prepare their drafts and the exposition of 
their design ideas after which these are uploaded onto MARPAS, a mobile augmented reality peer assessment 
system. The AR application constructs the relation between the image of the draft and the exposition. 
Subsequently, during the assessment process, the teacher designs two activities in which students have to mark 
peers’ work in an indoor environment as well as in an outdoor environment. These drafts are printed out as tags 
and posted in an exhibition and on an outside wall. The assessment situations in indoor and outdoor 
environments and the interfaces of MARPAS are presented in Figure 3. 


In the indoor situation, assessors go to the exhibition to capture the tag through their own camera on 
handheld devices. Then, they can see the work can be constructed as a physical produce in the real world. The 
introduction of assessees’ work and the assessment criteria can be shown on the device at the same time. In the 
outdoor situation, assessors go to outside to capture the tag on the wall, and then the visual work is presented on 
the wall in the real world. Assessors can review the assessees’ information and assessment criteria as well. 
However, assessment criteria are different based on the varied situations. For example, assessment criteria 
including suitable, originality and colour scheme have to be considered in the indoor assessment, and assessors 
mark the dimensions of suitable, exquisite level and usability in the outdoor assessment according to the 
features of outdoor situation. By this way assessors can mark the usability of the work at the current location. In 
addition, all assessment related to the work can be presented simultaneously on the device as shown as Figure 
4, and therefore assessors are able to review other assessors’ assessment and assessees can receive the 
assessment results. 



Figure 3. The Demonstration of the MARPAS 
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Eigure 4. The Representation of Assessment Results 

According to the different surroundings, students are not only able to acquire the relative explanation and 
representation of work but also apply appropriate assessment criteria that produce sufficient assessment results 
to mark peers’ work. MARPAS facilitates students to observe other assessors’ marking as well as receive 
assessment feedback. Therefore, students can reflect upon their work according to the various and meaningful 
feedback received. 


5. CONCLUSIONS 

This study has presented a framework for providing intelligent and mobile supports to enrich peer assessment. 
The limitation of time, space and devices can be eliminated. In this framework, students can review and assess 
peers' work represented with AR technology through combining virtual objects with the real world. Mobile AR 
technology provides flexible mobility and location-based adaptive contents to interact with the assessing work 
and the real world for individual students. Students can bring their own handheld devices to capture and acquire 
appropriate information at the right time in the right situation. By incorporating the techniques of AR, the 
proposed framework enables students to review peers' work in various ways and students can receive the 
assessment results immediately. The difficulty of reviewing peers' work and understanding peers' thinking can 
be resolved and sufficient information representation enables accurate assessment. In addition, appropriate 
assessment criteria and rich feedback facilitate students to reflect upon their own work and improve the quality 
of their work. Although the proposed framework has indicated the assistance of incorporating AR in peer 
assessment, considerable work remains to be done, including further large-scale classroom experiments and 
system adaptability. 
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